3 research outputs found
Recommended from our members
Understanding Internal Feature Development in Deep Convolutional Neural Networks for Time Series
Taken as a black box, a neural network applies a complex nonlinear function to its input to produce an output. This complex nonlinear function is determined by the network's internal weights. During the network's training process, these weights are adjusted based on a sample of input-output pairs so that the difference between the network's actual outputs and desired outputs is minimized. Deep neural networks have been employed very successfully as prediction models in many applications. A major remaining challenge is to understand how the internal adaptations that result from learning enable such models to make accurate predictions.
In this research, we propose several methods for interpreting the learned internal representations in neural networks in the context of time-series signal classification. Our two main interpretation goals are: (1) Feature Interpretation, to understand what features of the inputs are learned by the network's internal units; and (2) Feature Development, to visualize how the ability of individual network layers to differentiate among classes varies with the depth of the layer within the network.
To evaluate the proposed methods, we develop neural networks for sleep stage classification. The networks take as inputs physiological signals of human subjects during sleep and map these multidimensional time-series to sequences of symbols corresponding to different sleep stages. Our results demonstrate that our techniques succeed in our aims of Feature Interpretation and Feature Development. We show that the networks' internal units can learn features that closely resemble those used by human sleep experts in the traditional sleep stage scoring process, such as sleep spindles, K-complexes, and slow waves. Furthermore, our results describe the development of these features with layer depth, showing that the network assembles them gradually, as layer depth increases, by extracting simple building blocks in shallow layers and combining them in deeper layers to form more complex features. Additionally, we observe an increase in the ability of the network layers to differentiate among sleep stages as depth of the layers within the network increases
Recommended from our members
UCT-Enhanced Deep Convolutional Neural Networks for Move Recommendation in Go
Deep networks have been proved to be useful in predicting moves of human Go experts. Combining Upper Confidence bounds applied to Trees (UCT) with a large deep network creates an even more powerful AI in playing Go. Our project introduced a new feature, board patterns at the end of a game, used as inputs to the network. By adding the new feature, our model correctly predicted the experts’ move in 18% of the positions, compared to 6% without this feature. In practice, although the board pattern at the end of the game is invisible to Go players until the end of the game, collecting statistics during each simulation in UCT can approximate the board pattern at the end of game. With the dataset generated by UCT simulation, the network correctly predicted the experts’ move in 9% of positions
Recommended from our members
Flood damage in Bangkok: disaster an opportunity for creative destruction
We attempt to evaluate the paths to recovery following the flood in 2011 in Thailand, which severely damaged the economy. We use system dynamic to simulate the impact of flood and test the performance of the recovery effort. We set the boundary of our study to Bangkok. We build on Saeed’s model of Schumpeter’s concept of creative destruction, which he has posited as fore-runner to Forrester’s Urban Dynamics model. We extend Saeed’s model to subsume the infrastructure aging chain, land constraint, taxation, and service provision. We study the recovery policies implemented by Thai government as well as those alluded to in Urban Dynamics. We find that encouraging new investment and reducing cost of capital, paired with demolition of old infrastructure, help facilitate the recovery process